Sigma Delta Quantized Networks
نویسندگان
چکیده
Deep neural networks can be obscenely wasteful. When processing video, a convolutional network expends a fixed amount of computation for each frame with no regard to the similarity between neighbouring frames. As a result, it ends up repeatedly doing very similar computations. To put an end to such waste, we introduce Sigma-Delta networks. With each new input, each layer in this network sends a discretized form of its change in activation to the next layer. Thus the amount of computation that the network does scales with the amount of change in the input and layer activations, rather than the size of the network. We introduce an optimization method for converting any pre-trained deep network into an optimally efficient Sigma-Delta network, and show that our algorithm, if run on the appropriate hardware, could cut at least an order of magnitude from the computational cost of processing video data.
منابع مشابه
Approximating a bandlimited function using very coarsely quantized data: A family of stable sigma-delta modulators of arbitrary order
متن کامل
Sigma-Delta quantization of sub-Gaussian frame expansions and its application to compressed sensing
Suppose that the collection {ei}i=1 forms a frame for R, where each entry of the vector ei is a sub-Gaussian random variable. We consider expansions in such a frame, which are then quantized using a Sigma-Delta scheme. We show that an arbitrary signal in R can be recovered from its quantized frame coefficients up to an error which decays root-exponentially in the oversampling rate m/k. Here the...
متن کاملApproximating a Bandlimited Function Using Very Coarsely Quantized Data: Improved Error Estimates in Sigma-delta Modulation
This paper concerns fine analytical error estimates in analog-to-digital conversion of bandlimited functions using the method of sigma-delta modulation (also called Σ∆ quantization). This method has found widespread usage in practice due to several advantages in its implementation compared to conventional methods (see [1, 11]). A recent mathematical treatment of this problem appears in [3]. Bel...
متن کاملGeneralized sigma-derivation on Banach algebras
Let $mathcal{A}$ be a Banach algebra and $mathcal{M}$ be a Banach $mathcal{A}$-bimodule. We say that a linear mapping $delta:mathcal{A} rightarrow mathcal{M}$ is a generalized $sigma$-derivation whenever there exists a $sigma$-derivation $d:mathcal{A} rightarrow mathcal{M}$ such that $delta(ab) = delta(a)sigma(b) + sigma(a)d(b)$, for all $a,b in mathcal{A}$. Giving some facts concerning general...
متن کاملTime-Mode Signal Quantization for Use in Sigma-Delta Modulators
The rapid scaling in modern CMOS technology has motivated the researchers to design new analog-to-digital converter (ADC) architectures that can properly work in lower supply voltage. An exchanging the data quantization procedure from the amplitude to the time domain, can be a promising alternative well adapt with the technology scaling. This paper is going to review the recent development in t...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- CoRR
دوره abs/1611.02024 شماره
صفحات -
تاریخ انتشار 2016